Validation API core #8348
Validation API core #8348
Conversation
…sk creation (put chunk creation into the end), need to update chunk generation approach to to per job
|
@klakhov ,
Hm, I think yes, it should produce a validation error during task creation.
I think it will be ether in a separate PR or in the PR with allocation reports, it seems to be the most relevant place. |
|
@coderabbitai review |
✅ Actions performedReview triggered.
|
cvat/apps/dataset_manager/task.py
Outdated
| if data.tracks and db_data.validation.mode == models.ValidationMode.GT_POOL: | ||
| # Only tags and shapes can be used in tasks with GT pool | ||
| raise ValidationError("Tracks are not supported when task validation mode is {}".format( | ||
| models.ValidationMode.GT_POOL | ||
| )) |
There was a problem hiding this comment.
So, as far as I understand we cant use tracks in tasks with HP.
But when I try to add a track in UI I get this error:
Seems error message is wrong
There was a problem hiding this comment.
@zhiltsov-max What is the nature of such restriction?
I understand that tracks with interpolation do not make sense in honeypot jobs, but is it the only reason?
There was a problem hiding this comment.
Frames are in the random order, tracks make no sense in such case. If the frames were ordered, honeypots could be (a) clearly identifiable or (b) could not be reused multiple times. Simple GT job doesn't have such a problem, so it can be used if frame ordering or tracks are required in the task.
There was a problem hiding this comment.
It makes sense if each frame of the track is a keyframe.
There was a problem hiding this comment.
Just do not understand the reason why we forced this restriction.
If a user does it, why not. It is the user's decision.
|
Motivation and context
Depends on #8272
Depends on #8321
random_per_job, which guarantees each annotation job gets the specified GT overlap, making each annotation job validatableServer API changes:
validation_modefield, reflecting the current validation configuration (immutable)validation_paramsfield, which allow to enable GT / GT_POOL validation for a task on its creationTasks with Honeypots
This validation mode affects task creation, so can only be used in task creation. It cannot be disabled or changed after the task is created. When honeypots are configured, each job in the task gets several extra validation frames.
The pool of available frames and the number of validation frames per job are specified by the user at task creation.
Limitations:
randomframe ordering.job_frame_mappingandoverlapcannot be used in such tasks.Honeypot frames and GT annotations are accessible via the GT job, as in the case with regular GT jobs. However, unlike regular tasks with GT jobs, task annotation import affects the GT job as well in tasks with honeypots. Task annotation export contains only GT annotations on validation frames (so, only the GT copy of validation frames is included).
How has this been tested?
Checklist
developbranch(cvat-canvas,
cvat-core,
cvat-data and
cvat-ui)
License
Feel free to contact the maintainers if that's a concern.
Summary by CodeRabbit
New Features
Bug Fixes
Documentation
Chores